ExaMind: A Multimodal AI-Based Online Examination System with Behavioral Analysis and Performance Evaluation

Abstract

The rapid adoption of online examination systems has introduced significant challenges in ensuring academic integrity and effective performance evaluation. Conventional systems primarily rely on basic monitoring techniques such as webcam surveillance and browser control, which are insufficient for capturing deeper behav- ioral patterns including student engagement, attention, and cognitive effort. To address these limitations, this paper presents ExaMind, a multimodal AI-based online examination system that integrates real-time monitoring with be- havioral performance analysis. The proposed system combines facial authentication, emotion recognition, head pose estimation, object detection, and mouse interaction tracking to analyze student behavior during examina- tions within a client–server architecture, where a PyQt-based examination interface captures interaction data and a Django-based backend processes and analyzes the collected information. The system classifies student behav- ior based on response time and correctness, enabling the identification of problem-solving patterns, and employs a weighted performance evaluation model that integrates accuracy, emotional state, and behavioral analysis to generate comprehensive performance metrics. Furthermore, the system provides detailed analytical dashboards for both students and instructors, enhancing feedback and evaluation quality. Experimental results demonstrate that the proposed approach improves examination integrity while offering meaningful insights into student engage- ment and learning behavior, thereby contributing to the development of intelligent and adaptive online assessment systems.

Introduction

The rapid growth of online learning has increased the use of digital examination platforms, but maintaining academic integrity and accurately assessing student performance remain significant challenges. Traditional online proctoring methods such as webcam monitoring, screen recording, and browser restrictions can detect visible misconduct but provide limited insights into student engagement, attention, confidence, and cognitive effort.

To address these limitations, the paper proposes ExaMind, a multimodal AI-powered online examination system that combines real-time monitoring with behavioral performance analysis. Unlike conventional systems that focus mainly on cheating detection, ExaMind integrates multiple data sources to provide a more comprehensive evaluation of student performance and behavior.

Key Features of ExaMind

The system combines several AI and computer vision modules:

• Face Authentication – verifies student identity using facial recognition.
• Emotion Detection – identifies emotional states such as focus, confusion, or distraction.
• Head Pose Estimation – monitors student attention and activity.
• Object Detection – detects unauthorized objects or additional persons.
• Mouse Tracking – records cursor movements, clicks, hover time, and interaction patterns.
• Behavioral Analysis – evaluates problem-solving behavior based on interaction data.

System Architecture

ExaMind follows a client–server architecture:

• A PyQt desktop application conducts the examination, records responses, captures mouse interactions, and collects webcam data.
• A Django-based backend processes the collected data, performs AI-based analysis, and stores information in a MySQL database.
• The system generates detailed reports for students and instructors, including engagement metrics and behavioral insights.

Methodology

The system collects and synchronizes:

• Mouse movements, clicks, hover duration, and response times.
• Facial expressions, head pose data, and environmental observations through webcam analysis.

Pretrained deep learning models such as YOLOv8 for object detection, MediaPipe for head pose estimation, and ResNet18 for emotion detection are used to analyze the data.

Behavior Classification

Student responses are categorized based on answer correctness and response time:

This classification helps identify engagement levels, confidence, and problem-solving patterns.

Performance Evaluation Model

Student performance is assessed using three metrics:

1. Accuracy Score (50%) – based on the percentage of correct answers.
2. Emotion Score (30%) – derived from positive emotional states detected during the exam.
3. Behavior Score (20%) – calculated from the distribution of behavioral categories.

The final score is computed as:

Overall Score = 0.5 × Accuracy + 0.3 × Emotion Score + 0.2 × Behavior Score

Results and Contributions

Experimental evaluation showed that ExaMind successfully captures and analyzes multimodal data during examinations. By combining academic performance, emotional state, and behavioral patterns, the system provides a more holistic assessment than traditional score-based online examination systems.

Conclusion

This paper presented ExaMind, a multimodal AI-based online examination system designed to enhance both examina- tion integrity and performance evaluation. The system integrates multiple monitoring techniques, including facial authentica- tion, emotion detection, head pose estimation, object detection, and mouse interaction tracking, within a unified client–server architecture. By combining a PyQt-based examination interface with a Django-based backend, the system enables efficient real-time data collection, processing, and analysis, ensuring a structured and reliable examination environment. Unlike traditional online examination systems that rely solely on final scores, the proposed approach incorporates be- havioral analysis and a weighted performance evaluation model that considers accuracy, emotional state, and interaction pat- terns. This integration enables a more comprehensive understanding of student performance, capturing not only correctness but also engagement, attention, and cognitive effort. The system further enhances usability by providing detailed analytical dashboards for both students and instructors, supporting improved feedback mechanisms and data-driven decision-making in educational environments. Experimental observations indicate that the proposed system effectively captures meaningful behavioral patterns and generates insightful performance metrics. The integration of multimodal data improves the robustness and reliability of evaluation compared to conventional systems that rely on single-modality analysis.

References

[1] A. W. Muzaffar, M. Tahir, M. W. Anwar, Q. Chaudry, S. R. Mir, and Y. Rasheed, “A Systematic Review of Online Exams Solutions in E-Learning: Techniques, Tools, and Global Adoption,” IEEE Access, vol. 9, pp. 32689–32712, 2021. [2] R. Ryu, S. Yeom, D. Herbert, and J. Dermoudy, “A Comprehensive Survey of Context-Aware Continuous Implicit Authentication in Online Learning Environments,” IEEE Access, vol. 11, pp. 24561–24585, 2023. [3] N. G. Niharika and S. N. Nayak, “Artificial Intelligence Based Online Examination Proctoring System,” IJRASET, vol. 11, no. IX, pp. 569–575, Sep. 2023. [4] S. Satre, S. Patil, T. Mane, V. Molawade, T. Gawand, and A. Mishra, “Online Exam Proctoring System Based on Artificial Intelligence,” Proc. IEEE Int. Conf., 2023. [5] A. Khaesawad, Y. Fukuchi, V. Yem, and N. Nishiuchi, “Machine Learning-Based Classification of Programming Logic Understanding Levels by Mouse-Tracking Heatmaps,” IEEE Access, vol. 13, pp. 89905–89918, 2025. [6] Gallagher, J. E., and E. J. Oughton, “Surveying You Only Look Once (YOLO) Multispectral Object Detection Advance- ments, Applications, and Challenges,” IEEE Access, vol. 13, pp. 7366–7395, 2025. [7] Ferraris, C., G. Amprimo, S. Cerfoglio, L. Vismara, and V. Cimolin, “A Deep Dive Into MediaPipe Pose for Postural Assessment: A Comparative Investigation,” IEEE Access, vol. 13, pp. 211055–211074, 2025. [8] S. Mishra, V. Verma, N. Akhtar, S. Chaturvedi, and Y. Perwej, “An Intelligent Motion Detection Using OpenCV,” Int. J. Sci. Res. Sci. Eng. Technol., vol. 9, no. 2, pp. 51–63, 2022. [9] X. Chen, X. Hu, Y. Huang, H. Jiang, W. Ji, Y. Jiang, and L. Zhang, “Deep Learning-Based Software Engineering: Progress, Challenges, and Opportunities,” Science China Information Sciences, vol. 68, no. 1, 2025. [10] Z. B. Alawi, “A Comparative Survey of PyTorch vs TensorFlow for Deep Learning: Usability, Performance, and De- ployment Trade-Offs,” arXiv preprint arXiv:2508.04035, 2025. [11] Zhao, F., Zhang, C., and Geng, B. (2024). Deep multimodal data fusion. ACM computing surveys, 56(9), 1-36. [12] Liu, H., Brailsford, T., and Bull, L. (2024, October). Resnet18 performance: Impact of network depth and image resolution on image classification. In Proceedings of the 2024 8th international conference on advances in artificial intelligence (pp. 351-356). [13] Fu, Q. (2024). Research on student behavior analysis and grade prediction system based on student behavior character- istics. Scalable Computing: Practice and Experience, 25(1), 217-228. [14] Sohan, M., Sai Ram, T., and Rami Reddy, C. V. (2024). A review on yolov8 and its advancements. In International conference on data intelligence and cognitive informatics (pp. 529-545). Springer, Singapore. [15] Tiwari, S. (2025). Advancing Client Risk Scoring: From Rule-Based Systems to Machine Learning Approaches. Journal of Computer Science and Technology Studies, 7(8), 01-07. [16] Jin, C., Xiao, Y., Wu, H., Ji, X., Li, G., Shuai, J., ... and Xiong, L. (2025). Human-model interaction-based decision support system for optimizing food safety assessment. Food Research International, 208, 116156. [17] Nissinen, M., Silvennoinen, E., and Saqr, M. (2023). How assessment analytics can help to improve reliability, effi- ciency, and fairness of entrance examinations. [18] A. S. Aruna, K. R. Babu, and K. Deepthi, “A Deep Drug Prediction Framework for Viral Infectious Diseases Using an Optimizer-Based Ensemble of CNN: COVID-19 as a Case Study,” Molecular Diversity, vol. 29, no. 3, pp. 2473–2487, 2025. [19] Kayumova, M., Akbarova, S., Azizova, M., Hojiyeva, I., Karimova, F., and Makhmudova, F. (2024, December). Data driven teaching and real time decision making in education management. In Proceedings of the 8th International Con- ference on Future Networks and Distributed Systems (pp. 756-763). [20] Vashishth, T. K., Sharma, V., Sharma, K. K., Kumar, B., Panwar, R., and Chaudhary, S. (2024). AI-driven learning analytics for personalized feedback and assessment in higher education. In Using traditional design methods to enhance AI-driven decision making (pp. 206-230). IGI Global Scientific Publishing. [21] Shnitzer, T., Ou, A., Silva, M., Soule, K., Sun, Y., Solomon, J., ... and Yurochkin, M. (2023). Large language model routing with benchmark datasets. arXiv preprint arXiv:2309.15789. [22] Rafik, H., Maizate, A., and Ettaoufik, A. (2023). Data Security Mechanisms, Approaches, and Challenges for e-Health Smart Systems. International Journal of Online and Biomedical Engineering, 19(2). [23] Maulani, I. E., Azis, I., Cahya, M. N., and Komarudin, K. (2024). Implementation Of Object-Oriented Programming With Pyqt: Development Of Calculation Application. Devotion: Journal of Research and Community Service, 5(1), 156-163. [24] Thakur, P., and Jadon, P. (2023, May). Django: Developing web using python. In 2023 3rd International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE) (pp. 303-306). IEEE. [25] Almalawi, A., Soh, B., Li, A., and Samra, H. (2024). Predictive models for educational purposes: A systematic review. Big Data and Cognitive Computing, 8(12), 187. [26] Mienye, I. D., Swart, T. G., and Obaido, G. (2024). Recurrent neural networks: A comprehensive review of architectures, variants, and applications. Information, 15(9), 517. [27] Ghojogh, B., and Ghodsi, A. (2023). Recurrent neural networks and long short-term memory networks: Tutorial and survey. arXiv preprint arXiv:2304.11461. [28] Gligorea, I., Cioca, M., Oancea, R., Gorski, A. T., Gorski, H., and Tudorache, P. (2023). Adaptive learning using artificial intelligence in e-learning: A literature review. Education Sciences, 13(12), 1216.

Copyright

Copyright © 2026 Alan C B, Ansha Anilkumar E, Nandana P V, Pooja T K, Aruna A S. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.